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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

Responsive to communication(s) filed on 01 February 2000 , 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) M is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1^4 is/are rejected. 

7) [3 -Claim(s) 1 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) £3 The drawing(s) filed on 01 February 2000 is/are: a)^ accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

: 1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Oath/Declaration 

1 . The oath or declaration is defective. A new oath or declaration in compliance 

with 37 CFR 1.67(a) identifying this application by application number and filing date is 

required. See MPEP §§ 602.01 and 602.02. 

The oath or declaration is defective because: 

It does not identify the citizenship of each inventor. 

Claim Objections 

2. Claim 1 is objected to because of the following informalities: based on page 2, 
line 16 of the specification, the word "coarse" was misspelled as "course". Appropriate 
correction is required. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4: Claim 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
the article "MPEG-4: An Object-Based Multimedia Coding Standard Supporting Mobile 
Applications," by Puri et al. (hereinafter called Puri) in view of the article "Recent Efforts 
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Toward Graphics Standardization," by Newman and Van Dam (hereinafter called 
Newman). Puri discloses the following: 

A method of coarse representation of the shape of a visible object (i.e. shape 
representation, p. 12, col. 2, line 28) in a digital picture (i.e. MPEG-4 video, p. 12, 
col. 2, line 4) comprising the steps of: 

Segmenting visible objects from the digital picture (p. 12, col. 2, lines 28- 
31; p. 13, fig. 5); 

Extracting a bitmap (i.e. binary shape mask, p. 12, col. 2, lines 30-31) for 
an object of interest from the segmented visible objects; and 
Estimating from the bitmap a binding box (i.e. bounding box, p. 14, col. 1 , 
lines 21-24; p. 14, fig. 10) for the object of interest. 
5. Puri does not disclose the use of a display aspect ratio to estimate the size of a 
binding box. However, Newman points out that in a computer graphics system "one 
might have to redefine the size of the displayed image because of a change in the 
screen's aspect ratio" (p. 373, col. 2, lines 5-7). Therefore, it would have been obvious 
to one skilled in the art at the time of the invention to modify the method described by 
Puri to include the idea disclosed by Newman because it would result in improved 
portability across various types of displays (p. 373, col. 1, lines 13-15). It is noted that 
the field of computer graphics is closely related to digital video, and the concept of 
aspect ratio is equally relevant to both fields. 
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6. Claim 2, which is dependent on claim 1 , is rejected as being unpatentable over 
Puri in view of Newman and further in view of Blumberg (U.S. patent no. 6,449,639). 
Blumberg discloses the following, which is not disclosed by Puri or Newman: 

An estimating step comprising: 

Estimating in pixel units a set of parameters (i.e. values, col. 5, lines 48-52) for 
the binding box (i.e. rectangular portion of an image, col. 5, lines 46-48); and 
Normalizing the pixel units to form a feature vector representing the binding box 
(col. 5, lines 46-55). It is understood that the set of normalized values described 
by Blumberg is equivalent to a feature vector representing a bounding box. 

7. It would have been obvious to one skilled in the art at the time of the invention to 
modify the method described by Puri by including the method of Blumberg because this 
would achieve scale and resolution invariance (col. 5, lines 23-25). 

8. Claim 3, which depends on claim 2, is rejected as being unpatentable over Puri 
in view of Newman and Blumberg and further in view of Jain (U.S. patent 5,893,095). 
Jain discloses the following, which is not revealed by Puri, Newman, and Blumberg: 

The step of searching a video database (i.e. query, col. 9, lines 54, 64; fig. 1A ( 
modules 106, 108; database... such as.. .digital video, col. 9, lines 32-33; see 
also lines 1-2 of the abstract) having visible objects (i.e. visual objects, col. 9, 
lines 41-47), each visible object having an associated feature vector (col. 8, lines 
1-5), to find those visible objects whose feature vectors match the feature vector 
of the object of interest (col. 12, lines 58-60). 



Application/Control Number: 09/494,514 Page 5 

Art Unit: 2625 

9. It would have been obvious to one skilled in the art at the time of the invention to 
modify the method disclosed by Puri by including the method of Jain because this would 
provide easy and efficient access to the object in the digital picture as disclosed in claim 
1, based on a coarse description of the object (col. 1, lines 33-39, lines 51-57; col. 2, 
lines 54-61; col. 3, lines 53-56). 

10. Claim 4, which depends on claim 3, is rejected as being unpatentable over Puri 
in view of Newman, Blumberg, and Jain, and further in view of Crabtree (U.S. patent no. 
6;185,314). Jain reveals the following: 

A searching step comprising the steps of: 

Computing distances (col. 7, lines 11-12) according to a specified distance metric 
(coL 8, lines 20-23) between primitive vectors or feature vectors for visible 
objects in the video database (col. 12, lines 63-64); 

Sorting the distances in descending order to produce a sort list of feature vectors 
and associated visible objects (col. 26, lines 57-58; col. 11, lines 41-52); and 
Displaying the visible objects associated with the feature vectors that are at the 
top of the sort list (fig. 4; col. 11, lines 35-41; col. 13, lines 21-29). 

1 1 . Puri, Newman, Blumberg, and Jain do not reveal the step of computing aspect 
ratios for all visible objects in a video database. However, Crabtree discloses a tracking 
system in which features such as bounding box aspect ratio are derived for each object 
or region cluster in a video sequence for the purpose of identification (col. 19, lines 31- 
33, 41-43). It would have been obvious to one skilled in the art at the time of the 
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invention to modify the method of Jain by including the aspect ratio step disclosed by 
Crabtree because this would provide an easily calculated object feature that can be 
used for initial comparison of objects (col. 21, lines 5-10). 

Other Sources 

12. The book Digital Image Processing by Gregory Baxes describes image 
segmentation (p. 124) and feature extraction (p. 153), including shape measurements 
such as aspect ratio (p. 158). The use of a bounding box is also shown (fig. 5.32f, p. 
156). In addition, normalization of shape measurements for the purpose of scaling 
invariance is described (p. 173). Bitmap boundary descriptions of objects are described 
as well (pp. 160-167). 

13. The method described in claims 3 and 4, namely searching a video database for 
objects in images based on feature vectors, calculating an abstract distance based on 
object features, and sorting and displaying the results, is well known in the art relating to 
content-based image retrieval (CBIR). In addition to Jain, see also Barber (U.S. patent 
5)751,286). 

14. The aspect ratio of a normalized bounding box, which relates to claims 3 and 4, 
is often used for handwriting recognition and text retrieval from image documents 
(optical character recognition or OCR). For example, see Chen (U.S. patent 5,438,630; 
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col. 9, lines 4-5; fig. 6) and Beernink (U.S. patent 5,615,285, col. 9, lines 40-41; fig. 8, 
step 164; fig. 10, step 210). 

1 5. The aspect ratio of a bounding box is also used for feature extraction and object 
classification in the context of face detection (Abdel-Mottaleb et al., U.S. patent 
6,263,113), detection of road signs (Laumeyer et al., U.S. patent 6,266,442; col. 5, lines 
24-26; col. 6, lines 3-9, 62-64), and tracking a moving object (Bianchi, U.S. patent 
5,434,617, col. 2, lines 18-21; fig. 13, items 410 and 1210). 

16. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael W. Bowen whose telephone number is 
(571)272-5969. The examiner can normally be reached on M-F 8AM-5:30PM. 

17. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571)272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is (571)273-8300. 

18. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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